gh-144995: Optimize memoryview == memoryview by vstinner · Pull Request #144996 · python/cpython

vstinner · 2026-02-19T11:50:58Z

Issue: Optimize memoryview comparison: a memoryview is equal to itself #144995

vstinner · 2026-02-19T11:52:10Z

Results of the benchmark from the issue:

bytes 0.000122 seconds
mview 0.000146 seconds
⇒ 1.197965 time slower

memoryview comparison complexity is no longer O(n) but O(1): values are no longer compared.

eendebakpt · 2026-02-19T14:51:56Z

Objects/memoryobject.c

 }

+static int
+is_float_format(const char *format)


Does this cover the complex types?

import numpy as np a = np.array([1+2j, 3+4j, float('nan')], dtype=np.complex128) mv = memoryview(a) mv == mv # False

This memory format is Zd. Oh, my change doesn't work for this memoryview. I should replace the blocklist with an allowlist. I'm not a memoryview/buffer expert. I didn't know that 3rd party projects can have their own format.

vstinner · 2026-02-19T15:53:04Z

@eendebakpt: I updated the PR to allow formats known to be safe for pointer comparison (integer types), instead of blocking formats known to use floats.

I excluded the format P since I don't know well this format. Or can we allow it?

Objects/memoryobject.c

eendebakpt · 2026-02-19T21:55:48Z

@eendebakpt: I updated the PR to allow formats known to be safe for pointer comparison (integer types), instead of blocking formats known to use floats.

I excluded the format P since I don't know well this format. Or can we allow it?

I think adding the P is fine (but I am no expert either). Leaving it out is the safe option, we can reconsider if this turns out to be a performance bottleneck.

Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>

Objects/memoryobject.c

Lib/test/test_memoryview.py

serhiy-storchaka · 2026-02-20T18:49:42Z

Lib/test/test_memoryview.py

+        # A memoryview is equal to itself: there is no need to compare
+        # individual values. This is not true for float values since they can
+        # be NaN, and NaN is not equal to itself.
+        for int_format in 'bBhHiIlLqQ':


Can "?" be tested? Can format starting with "@" be tested? Can the null format be tested?

I don't know how to test these formats. array.array doesn't support "P" and "?" formats and it doesn't support "@" byte order. Do you have an idea how to test these cases?

memoryview.cast() supports them.

Surprisingly:

>>> memoryview(b'\0\1').cast('?') == memoryview(b'\0\2').cast('?') False

even if

>>> list(memoryview(b'\0\1').cast('?')) == list(memoryview(b'\0\2').cast('?')) True

But this may be platform depending, so I would not test values different than 0 and 1. Or 1 is also not safe?

It may be undefined behavior to interpret random values except 0 as void* (even if it works on x86). Maybe there is a way to create an array of pointers in ctypes? Or it is not worth to bother?

@b

* Optimize also "P" format * Test also "m != m" * Handle native formats such as "@b"

vstinner · 2026-02-21T12:34:57Z

I updated the PR to address @serhiy-storchaka's review:

Optimize also "P" format
Test also "m != m"
Handle native formats such as "@B"

vstinner · 2026-02-21T15:25:35Z

I added tests on 4 more formats: @b, @b, P and ?.

serhiy-storchaka

I have some doubts about the 'P' test. It may be an operation with undefined behavior (although CPython may be never run on platforms were this does not work, but I am not sure). It would be safer to omit that test. There are no other tests for 'P' format. But the optimization should work for it (if we exclude undefined behavior).

Objects/memoryobject.c

vstinner · 2026-02-21T19:05:51Z

I modified the tests to check that the result with optimization is the same as the result without optimization:

        def check_equal(view, is_equal):
            self.assertEqual(view == view, is_equal)
            self.assertEqual(view != view, not is_equal)

            # Comparison with a different memoryview doesn't use
            # the optimization and should give the same result.
            view2 = memoryview(view)
            self.assertEqual(view2 == view, is_equal)
            self.assertEqual(view2 != view2, not is_equal)

I have some doubts about the 'P' test. It may be an operation with undefined behavior (although CPython may be never run on platforms were this does not work, but I am not sure). It would be safer to omit that test. There are no other tests for 'P' format. But the optimization should work for it (if we exclude undefined behavior).

For boolean (? format), I use memoryview(b'\0\1\2').cast('?') in the test. While m.tolist() == m.tolist() gives a different result than m == m, the important part here is that the can_compare_ptr optimization doesn't change m == m result.

If you are not confident with my P test and would prefer to remove the test, I would prefer removing the optimization for this type.

pythongh-144995: Optimize memoryview == memoryview

2294542

bedevere-app bot added the awaiting core review label Feb 19, 2026

bedevere-app bot mentioned this pull request Feb 19, 2026

Optimize memoryview comparison: a memoryview is equal to itself #144995

Open

eendebakpt reviewed Feb 19, 2026

View reviewed changes

vstinner added 2 commits February 19, 2026 16:45

Replace blocklist with allowlist

0dd7911

Dummy change to update GitHub

61e37e4

eendebakpt reviewed Feb 19, 2026

View reviewed changes

Objects/memoryobject.c Outdated Show resolved Hide resolved

eendebakpt reviewed Feb 19, 2026

View reviewed changes

Objects/memoryobject.c Outdated Show resolved Hide resolved

Apply suggestions from code review

102f26d

Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>

serhiy-storchaka reviewed Feb 20, 2026

View reviewed changes

Address review

2316fab

* Optimize also "P" format * Test also "m != m" * Handle native formats such as "@b"

Test more formats: @b, @b, P and ?

800e85f

serhiy-storchaka reviewed Feb 21, 2026

View reviewed changes

Objects/memoryobject.c Outdated Show resolved Hide resolved

vstinner added 2 commits February 21, 2026 19:52

Fix for empty format string

a754be4

Add tests with optimization disabled

1d72ed2

Uh oh!

Comments

Conversation

vstinner commented Feb 19, 2026 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner commented Feb 19, 2026

Uh oh!

eendebakpt Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

vstinner Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

vstinner commented Feb 19, 2026

Uh oh!

Uh oh!

Uh oh!

eendebakpt commented Feb 19, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

serhiy-storchaka Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

vstinner Feb 21, 2026

Choose a reason for hiding this comment

Uh oh!

serhiy-storchaka Feb 21, 2026

Choose a reason for hiding this comment

Uh oh!

serhiy-storchaka Feb 21, 2026

Choose a reason for hiding this comment

Uh oh!

vstinner commented Feb 21, 2026

Uh oh!

vstinner commented Feb 21, 2026

Uh oh!

serhiy-storchaka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vstinner commented Feb 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vstinner commented Feb 19, 2026 •

edited by bedevere-app bot

Loading